Named entity recognition and extraction using genetic programming

ABSTRACT

Disclosed herein are methods, systems, and apparatus, including computer programs encoded on computer storage media, for generating a pattern program using a genetic algorithm. The genetic algorithm operates on example data strings that represent the data categories to be recognized or extracted through named entity recognition. In the initialization stage, the initial pattern programs are generated based on example data strings that represent the data categories to be recognized or extracted through named entity recognition. Starting from the initial pattern programs, genetic operations are iteratively conducted to generate generations of offspring pattern programs. In each round of the genetic operation, offspring pattern programs are generated through the crossover operation and the mutation operation.

BACKGROUND

Advances in network and storage-subsystem design continue to enableincreasing scales of data streams be processed between and withincomputer systems. Meanwhile, the contents of such data streams aresubjected to ever increasing scrutiny. For example, the collection,analysis and storage of personal data are subject to scrutiny andregulation. Organizations have to ensure that personal data is gatheredlegally and under strict conditions. Organizations that collect andmanage personal data are obliged to protect it from misuse and illegalexploitation, and are obliged to respect the rights of data owners.Personal data or other sensitive data includes, but is not limited to,name, birth date, birth place, identification number, home address,credit card number, phone number, email address, URL, IP address, bankaccount number, etc.

Classification and extraction of personal data or other sensitive datafrom data streams involves named entity recognition. In general, namedentity recognition is a task of information extraction that seeks toidentify and classify atomic elements in texts into pre-definedcategories such as personal name, personal identification (for examplesocial security number “SSN” or resident identification number), homeaddress, email address, bank account number, phone number, credit cardnumber, etc. These pre-defined data categories are referred to as “namedentities”, or “entities” for short. The entities usually follow sometypes of syntactical patterns. Programs, e.g., regular expression,deterministic finite automata, or symbolic finite automata, are used tospecify patterns in data streams. However, generating such programstypically involves significant amount of expert programming efforts,which is inefficient and slow. In the era of big data and cloud-basedservices, service providers or platforms face the needs to tackle entityrecognition tasks on vast amount of varied categories of data streams,which cannot be handled by manual programming.

Therefore, there is a need to efficiently generate a program for namedentity recognition tasks.

BRIEF SUMMARY

This specification describes technologies for generating a patternprogram using a genetic algorithm. The genetic algorithm operates onexample data strings that represent the data categories to be recognizedor extracted through named entity recognition. Such example data stringsare referred to as “positive example” data strings. The geneticalgorithm may also operate on negative example data strings, whichrepresent data strings that negate the positive example data strings,e.g., not targets of the named entity recognition task. In theinitialization stage, the initial pattern programs are generated basedon example data strings that represent the data categories to berecognized or extracted through named entity recognition. Starting fromthe initial pattern programs, genetic operations are iterativelyconducted to generate generations of offspring pattern programs. In eachround of the genetic operation, offspring pattern programs are generatedthrough the crossover operation and the mutation operation. A smallportion of randomly generated pattern programs are added into eachgeneration of offspring pattern programs. A fitness function is used todetermine the fitness scores of the pattern programs in each generationof offspring pattern programs. The fitness scores are used to filter theoffspring pattern programs in a generation such that a population sizeof each generation of offspring pattern programs is maintained stable.For example, each generation includes a same number of offspring patternprograms. Upon the iterative genetic operations are completed, a patternprogram with a highest fitness score is selected for the named entityrecognition task.

If the genetic operations fail to generate a pattern program withdesirable extraction behavior, the example data strings are classifiedinto two or more sub-groups based on, e.g., the types or the lengths ofeach example data strings. The genetic operations are conducted on eachsub-group of example data strings in parallel, which each generates arespective pattern program. The multiple pattern programs are linkedthrough the “OR” function labels.

A fitness function includes one or more factors related to (1)conciseness of a pattern program; (2) a first matching rate of thepattern program on positive example data strings; (3) a second matchingrate of the pattern program on negative example data strings; or (4) anedit distance between the pattern program and a positive example datastring.

The genetic algorithm works on data strings that each exactly representsthe target data categories of named entity recognition. These technicalfeatures bring about valuable technical advantages. First, a patternprogram generated from the genetic algorithm will have a tailoredextraction behavior because the good “genes” contained in the exampledata strings are efficiently caught and carried over through the geneticoperations of the genetic algorithm. As such, the generated patternprogram will correctly detect and extract data strings of the targetdata categories. Further, using such example data strings also reduceshuman inputs and errors in the process because there is no need tomanually identify named entities from an unrepresentative data string.Also, the initial population of pattern programs is generated largely,e.g., 90%, from the example data strings, which substantially reducesthe amount of iterative genetic operations required to achieve asatisfactory pattern program. This savings in computing resources iscritical in managing large scale data streams in the era of big data andcloud-based data services.

Moreover, the fitness function considers whether a pattern programmatches a negative example data string, which is not a target of a namedentity recognition task. Resultantly, a pattern program selected basedon the fitness function will avoid data categories that are representedby the negative example data strings. Therefore, the false positiveerrors will be substantially reduced, which makes the outcomes of thenamed entity recognition tasks more reliable and meaningful. As such,the techniques of the specification are efficient and suitable forconducting named entity recognition tasks on large scale data streams.

This specification also provides one or more non-transitorycomputer-readable storage media coupled to one or more processors andhaving instructions stored thereon, which, when executed by the one ormore processors, cause the one or more processors to perform operationsin accordance with embodiments of the methods provided herein.

This specification further provides a system for implementing themethods provided herein. The system includes one or more processors, anda computer-readable storage medium coupled to the one or more processorshaving instructions stored thereon which, when executed by the one ormore processors, cause the one or more processors to perform operationsin accordance with embodiments of the methods provided herein.

It is appreciated that methods in accordance with this specification mayinclude any combination of the aspects and features described herein.That is, methods in accordance with this specification are not limitedto the combinations of aspects and features specifically describedherein, but also include any combination of the aspects and featuresprovided.

The details of one or more embodiments of this specification are setforth in the accompanying drawings and the description below. Otherfeatures and advantages of this specification will be apparent from thedescription and drawings, and from the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an example of an environment that canbe used to execute embodiments of this specification.

FIG. 2 is a diagram illustrating an example of an operation inaccordance with embodiments of this specification.

FIG. 3 is an example program generation module of generating patternprograms in accordance with embodiments of this specification.

FIG. 4 schematically shows an example process of generating patternprograms using genetic algorithms in accordance with embodiments of thisspecification.

FIG. 5 schematically shows another example process of generating patternprograms using genetic algorithms in accordance with embodiments of thisspecification.

Like reference numbers and designations in the various drawings indicatelike elements.

DETAILED DESCRIPTION

This specification describes technologies for generating a patternprogram using a genetic algorithm. The genetic algorithm operates onexample data strings that represent the data categories to be recognizedor extracted through named entity recognition. Such example data stringsare referred to as “positive example” data strings. The geneticalgorithm may also operate on negative example data strings, whichrepresent data strings that negate the positive example data strings,e.g., not targets of the named entity recognition task. In theinitialization stage, the initial pattern programs are generated basedon example data strings that represent the data categories to berecognized or extracted through named entity recognition. Starting fromthe initial pattern programs, genetic operations are iterativelyconducted to generate generations of offspring pattern programs. In eachround of the genetic operation, offspring pattern programs are generatedthrough the crossover operation and the mutation operation. A smallportion of randomly generated pattern programs are added into eachgeneration of offspring pattern programs. A fitness function is used todetermine the fitness scores of the pattern programs in each generationof offspring pattern programs. The fitness scores are used to filter theoffspring pattern programs in a generation such that a population sizeof each generation of offspring pattern programs is maintained stable.For example, each generation includes a same number of offspring patternprograms. Upon completion of the iterative genetic operations, a patternprogram with a highest fitness score is selected for the named entityrecognition task.

If the genetic operations fail to generate a pattern program withdesirable extraction behavior, the example data strings are classifiedinto two or more sub-groups based on, e.g., the types or the lengths ofeach example data strings. The genetic operations are conducted on eachsub-group of example data strings in parallel, which each generates arespective pattern program. The multiple pattern programs are linkedthrough the “OR” function labels.

A fitness function includes one or more factors related to (1)conciseness of a pattern program; (2) a first matching rate of thepattern program on positive example data strings; (3) a second matchingrate of the pattern program on negative example data strings; or (4) anedit distance between the pattern program and a positive example datastring.

The present specification is not limited to any particular embodiment,aspects, concepts, structures, functionalities or examples describedherein. Rather, any of the embodiment, aspects, concepts, structures,functionalities or examples described herein are non-limiting, and thepresent specification may be used in various ways that provide benefitsand advantages in computing, programming and data management in general.

FIG. 1 is an operation environment 100 for detecting data strings oftarget data categories from data streams, e.g., named entityrecognition. The environment 100 includes one or more users 110, aservice provider 120 and one or more sources of data streams 130, allcommunicating or logically connected to one another through a network140. Each of the user 110 or the service provider 120 is a computingdevice such as a personal computer “PC”, a server, a router, a networkPC, a mobile device, a peer device or other common network nodes. Thenetwork 140 may be one or more local area networks (“LAN”) and/or one ormore wide area networks (“WAN”) or other networks, which are configuredas enterprise-wide computer networks, intranets or internet or othernetwork applications. The user 110, the service provider 120 or thesource of the data streams 130 are categorized based on the respectivefunctions in the environment 100. The user 110, the service provider 120or a source of the data streams 130 may physically reside on a samecomputing device or in physically separated computing devices. The user110, the service provider 120 or a source of the data streams 130 maybelong to a same individual or legal entity, or different individuals orlegal entities. For example, a user 110 may be a business division of afinancial technology company and may have sensitive data detection,recognition, and scrutiny tasks to perform on the data streams 130. Theservice provider 120 may be a technology service department of the samefinancial technology company or a third party service provider thatprovides services on named entity recognition.

FIG. 2 shows a flow diagram of a process flow 200 of the operations orinteractions between/among the parties of the environment 100. Withreference to FIGS. 1 and 2 together, in example operation 210, a user110 designates or provides an initial set of example data strings to theservice provider 120. A data string includes a combination of one ormore letters, characters, signs, and/or other elements of expression.The initial set of example data strings contains examples that exactlyrepresent target data categories the user 110 desires to extract orrecognize from data streams 130. That is, an example data string doesnot contain any characters or data bits beyond the represented targetdata category.

A data stream 130 may be specific to a user 110 or may be shared orapplicable to multiple users 110. Similarly, a user 110 may use two ormore data streams 130. The user 110 may have a same or different namedentity recognition tasks for each data stream 130. For different datastreams 130, the user 110 may provide different initial sets of exampledata strings. As such, the user 110 may provide a named entityrecognition task to the service provider 120, which specifies theapplicable data stream 130 and the respective initial set of exampledata strings. The task may also specify the data categories of datastrings to be recognized. The specified data categories may already berepresented by the initial sets of example data strings or may berepresented by further example data strings as described herein. Forexample, the user 110 may require that personal data be recognized froma data stream 130. Example personal data includes a person's name, birthdate, birth place, identification number, home address, credit cardnumber, phone number, email address, URL, IP address, bank accountnumber, etc. In some embodiments, the user 110 provides example datastrings of personal data to the service provider 120. The example datastrings of personal data may include multiple categories of personaldata, e.g., phone number, personal identification number, credit cardnumber, etc. The example data strings of personal data may also includevarious data formats or pattern formats of a same category of personaldata. For example, example phone numbers may include patterns of:

001.234.456.7899;

+1.234.456.7899;

1 234 456 7899;

(234) 456 7899;

234 456 7899.

In some embodiments, the example data strings each exactly contain atarget data category itself. In some embodiments, at least some of theprovided example data strings each contain a first snippet representinga target data category and a second snippet that does not represent atarget category. The second snippet can provide a context forrecognizing of the target data category as contained in the first set ofsnippets. The user 110 may identify the first snippet as representing atarget data category. The user 110 may also specify the named entityrecognition task for the service provider to identify the first snippetas representing a target data category. In some embodiments, the exampledata strings may also include data strings representative of datacategories that are not targets for the named entity recognition tasksof a user 110. In the description herein, a “positive example datastring” refers to an example data string representative of a target datacategory. A “negative example data string” refers to an example datastring representative of a data category that is not a target for thenamed entity recognition tasks.

In example operation 220, the program generation module 122 generates anamed entity recognition program based on the example data stringsprovided by the user 110 using a genetic algorithm. The generated namedentity recognition program represents syntactical data patterns of thetarget data categories, and are referred to herein as “pattern program”for descriptive purposes. The pattern program may be in the form of aregular expression, a deterministic finite automata (“DFA”), a symbolicfinite automata (“SFA”) or other suitable programs representingsyntactical data patterns. In some embodiments, the pattern program isgenerated via a genetic algorithm implemented by the program generationmodule 122 of the service provider 120. In the description herein, aregular expression is used as an example pattern program to illustratethe operation of the service provider 120 and/or the program generationmodule 122.

In some embodiments, the program generation module 122 performs aninitializing operation, a synthesizing operation and a validationoperation. In the initializing operation, an initial population ofcandidate programs is obtained. In some embodiments, a majority of theinitial candidate programs are obtained based on the positive exampledata strings. For example, for each positive example data string, one ormore candidate regular expressions are obtained, whose extractionbehavior is consistent with a target data category represented by thepositive example data string. It should be appreciated that for eachtarget data category, multiple regular expressions can be generated. Insome embodiments, some candidate programs are generated randomly. Aratio between the number of candidate programs obtained based on thepositive example data strings and the number of randomly generatedcandidate programs is a parameter of the initializing operation, whichcan be adjusted. In some embodiments, the ratio is 9:1 such that thedesirable extraction behavior or good “genes” of the positive exampledata strings are readily caught and carried over in the geneticoperations. A population size, e.g., a total number of the candidateregular expressions, of the initial population of candidate regularexpressions is another adjustable parameter of the initializingoperation.

In the synthesizing or genetic operation, the initial candidate programsevolve through the operation of the genetic algorithm. The geneticalgorithm is implemented in an iterative manner. In each round of theevolution, the candidate programs in a parent population are synthesizedto create a child generation of candidate programs. The synthesizing mayinclude crossover and mutation operations. A ratio between the childcandidate programs generated from the crossover operations and the childcandidate programs generated from the mutation operations is a parameterof the synthesizing, which can be adjusted. In some embodiments, theratio is around 9:1 and may be adjusted to be larger or smaller than9:1. The candidate programs are each evaluated by a fitness function todetermine a fitness score. A fitness score indicates a degree that theextraction behavior of the pattern program is consistent with the targetdata categories represented by the example data strings or other datastrings used in the calculation of the fitness scores. In someembodiments, the example data strings that are used to generate theinitial candidate programs are also used to calculate the fitness scoresof the initial or child candidate programs. In some embodiments, theexample data strings that are provided by the user 110 are divided intotwo groups. One group of the example data strings are used to generatethe initial candidate programs and the other group of the example datastrings are used to calculate the fitness scores of the candidateprograms. The latter approach may help to avoid overfitting issues, ifany. In the description herein, for illustration purposes, the exampledata strings that are used to generate the initial candidate programsare also used to calculate the fitness scores of the candidate programs,which does not limit the scope of the specification.

In some embodiments, a candidate program's fitness score affects its usein the next round of the evolution, if any. For example, a new parentpopulation of the candidate programs may be selected based on thefitness scores of the existing candidate programs. For example, thecandidate programs with lower fitness scores may be filtered out as“unfit” to become parents for the next generation of evolution. In someexamples, a candidate program's probability of being selected forcrossover and/or mutation depends on the fitness score of the candidateprogram. For example, a candidate program with a higher fitness scorewill have a higher probability of mating with another candidate programin the crossover operation. A candidate program with a higher fitnessscore generally also has a higher probability of being selected for themutation operation, although the probability variation on the mutationoperation is less significant than the probability variation on thecrossover operation.

In some embodiments, the new population of parent programs is selectedonly from the most recent generation of child candidate programs. Insome embodiments, the new population of parent programs is selected froma pool of all existing candidate programs based on the fitness scores.For example, existing candidate programs of higher fitness scores areselected to form the new population of parent programs. In thedescription herein, a “parent generation” of candidate program refers toa generation of candidate programs that is used to generate newcandidate programs under the synthesizing operation; and a “childgeneration” of candidate program refers to a generation of candidateprograms that is generated from the synthesizing operation. A mostrecent child generation of candidate program may exactly overlap a newparent generation of candidate programs if the whole population of themost recent child generation is used for the next round of geneticoperation. A “generation” of candidate programs is used as applicable toeither one of a child generation or a parent generation of candidateprograms.

In some embodiments, a generation of candidate programs also includes apercentage, e.g., in a range between 5% and 15%, of randomly generatedcandidate programs. The mixture of candidate programs generated via thegenetic operation and the randomly generated candidate programs ensuresthat the “good genes” are maintained over generations and new “boostinggenes” be introduced. Resultantly, fitness scores of the candidateprograms are generally improved over generations. The synthesizingoperation completes when a fitness score of a candidate program meets afirst threshold value or a total number of the evolution rounds reachesa second threshold value. After the synthesizing operation completes, acandidate program with a highest fitness score is selected as a finalpattern program to be used in the named entity recognition task. Thefinal program may not necessarily be selected from the last generationof candidate programs. The final program may be selected from anygeneration of candidate programs.

The finiteness function may include various forms and criteria, whichare all included within the scope of the specification. In someembodiments, a fitness function includes factors related to concisenessof a candidate regular expression (e.g., a length of the candidateexpression); a first matching rate of the candidate regular expressionon the positive example data strings; a second matching rate of thecandidate regular expression on the negative example data strings; or anedit distance between the candidate regular expression and an exampledata string.

The first matching rate is calculated as a ratio between a number of thepositive example data strings that are 100% matched by the candidateregular expression and the total number of the positive example datastrings. The second matching rate is calculated as a ratio between anumber of the negative example data strings that are 100% matched by thecandidate regular expression and the total number of the negativeexample data strings. The edit distance is determined as the minimumnumber of edits that transfer an extracted data string into a targetdata category contained in a positive example data string. For example,in some embodiments, characters in an example data string that arematched by the extraction behavior of a candidate regular expression andthe characters in the example data string that are missed by theextraction behavior of the candidate regular expression are analyzed todetermine the edit distance of the candidate regular expression.

In example operation 230, the extraction module 124 uses the regularexpression generated by the program generation module 122 to extracttarget data categories from the data streams 130. Specifically, theextraction module 124 finds data strings in the data stream 130 thatmatches the patterns represented by the regular expression. In someembodiments, a percentage matching threshold may be used in implementingthe extraction operation. For example, if a data string in the datastream 130 includes characters or a snippet that matches the regularexpression by a percentage higher than 55%, the extraction module 124will extract the data string as belonging to the target data categories.The percentage matching threshold may be adjustable based on theconfiguration of the named entity recognition task, e.g., the toleranceon false positive or false negative designated by the user 110.

In example operation 240, the extracted or recognized data strings areprovided to the user 110. The user 110 may examine the providedextraction results and may confirm the correct extractions and mayidentify the false extractions, e.g., false positive or false negative.

In the learning operation 250, the user 110 provides the correctextractions and/or the false extractions as training set to the serviceprovider 120 to adjust the synthesizing operation. For example, thefalse positive extractions are used as additional negative example datastrings in training the candidate programs. The false negative results,i.e., data strings in the target data categories that are not extractedby the regular expression, may be provided as additional positiveexample data strings. With the additional example data strings, scoresof the candidate programs in various generation of the evolution may berecalculated, which changes the process and the results of thesynthesizing operation. In some embodiments, the synthesizing operationis not adjusted at the launching point thereof, and is trained startingat a generation in the middle of the evolution process. In someembodiments, the fitness scores are only recalculated for all thecandidate programs already generated in the evolution process, i.e.,without further synthesizing operations, such that the recalculatedfitness scores may lead to a different candidate program be selected asthe final program and no new candidate program is generated. Other usesof the additional negative example data strings or additional positiveexample data strings are also possible and included within the scope ofthe specification. For example, the initial candidate pattern programsmay be generated using different strategies than those initially used inthe operation 220. As a result, a new regular expression is generatedwhich has a higher fitness score than the previous regular expressionused in the operation 240. The fitness scores of the new regularexpression and the previous regular expression are calculated using sameexample data strings, e.g., at least one of the initial set of exampledata strings and the additional example data strings.

In some embodiments, the parameters of the genetic algorithm may beadjusted in the learning operation 250. For example, the ratio betweenthe candidate programs generated from the positive example data stringsand randomly generated candidate programs may be adjusted based on thefeedback from the user 110 on the extraction results. For example, ifthe false negatives are representative by the initial positive exampledata strings, the learning process may lower the percentage of randomlygenerated candidate programs in the genetic operations so that the“genes” of the positive example data strings are better represented inthe regular expression generated by the genetic operations.

FIG. 3 is an example program generation module 122. The programgeneration module 122 includes an initialization unit 310, a randomprogram generation unit 320, a synthesizing unit 330, a fitnessmeasurement unit 340, a controller 350, and a learning unit 360. Theinitialization unit 310 includes an example grouping unit 312, aninitial program generation unit 314, and a parse tree unit 316. Thesynthesizing unit 330 includes a crossover unit 332 and a mutation unit334. The controller 350 includes an adjustment unit 352.

In some embodiments, the program generation module 122 and the unitsthereof are computer executable instructions dedicated for respectivefunctions and operation. The executable instructions include routines,programs, objects, components, and data structures, which when executedby a processor, enable the processor to perform particular tasks orimplement particular abstract data types. The units of the programgeneration module 122 may reside on a same computing device or mayreside on multiple computing devices functioning together in adistributed computing environment. In a distributed computingenvironment, the units of the program generation module 122 may bestored in local or remote computer storage media including computermemory devices.

Operations and functions of the software units of the program generationmodule 122 are further described herein.

FIG. 4 shows an example process 400 of the program generation module 122generating a regular expression based on example data strings providedby the user 110. In some embodiments, the example data strings includepositive example data strings and negative example data strings. Thepositive example data strings each contains exactly an example targetdata category, e.g., an example birth date, an example social securitynumber, an example resident identification number, an example bankaccount number, etc. The use of the example target data categoriesthemselves as the positive examples data strings will simplify theoperation of the genetic algorithm and will facilitate that the “goodgenes” of the example target data categories be carried over in theevolution process. The negative example data strings each do not containa target data category. In some embodiments, some of the negativeexample data strings are counterexamples, which represent exceptions toa general characteristic of a target data category. For example,although data string “10.26.2050” appears to follow the data format of abirth date, the data string may be used as a counterexample of abirthdate because at the time of the operation, nobody has been born onthe year of 2050. In some embodiments, the counterexample data stringseach contain exactly a counterexample of a target data category withoutany other/additional character. Inclusion of such counterexamplesenables that the “bad genes” of the counterexamples be avoided in theevolution process of the genetic operations. The counterexample datastrings may be identified or classified separately from other negativeexample data strings. In some embodiments, the example data stringsinitially received from a user 110 do not include any negative exampledata strings. In some embodiments, the example data strings receivedfrom a user 110 include only positive example data strings andcounterexample data strings and do not include other negative exampledata strings.

The example process 400 is described with an example task of generatinga regular expression based on example data strings, for illustrativepurposes. In example act 410, the initialization unit 310 obtains aninitial population of candidate regular expressions, which is referredto as the zero generation G₀ for descriptive purposes. The example act410 includes sub-acts 412, 414 and 418. In sub-act 412, optionally, theexample grouping unit 312 may conduct an initial grouping of theexisting example data strings into initial groups with a goal thatexample data strings in a same initial group share a certain patterncharacteristic to be represented by a same regular expression. Forexample, the initial grouping may be conducted based on the characterclass of the example data strings, e.g., whether an example data stringcontains word characters without digits, whether an example data stringcontains digits without word characters, or whether an example datastring contains a mixture of digits and word characters. The initialgrouping may also consider the natural language of the word characters,e.g., whether the word characters are Chinese, English, Japanese orKorean. The initial grouping may also consider the language family ofthe word characters, e.g., whether the word characters belongs toCeltic, Italic, Sinitic, Germanic, Slavic or other language families.The initial grouping may also consider the relevant target datacategories represented by the example data strings. For example, exampledata strings of birth dates may include different formats like“mm-dd-yy”; “mm/dd/yy”; “dd-mm-yyyy”; “yyyy.mm.dd”; or other formats.The example data strings containing different formats of birth dateinformation are grouped together. The initial grouping may also considerthe length of the example data strings.

In some embodiments, the example grouping unit 312 assigns weights toeach groups of example data strings. The weights may impact the analysisof candidate programs in the fitness scores and the match rates. Theweights may also affect the amount of example data strings of each groupthat are actually used in the genetic operations. The weights assignedto each group of example data strings are adjustable, dynamically in thegenetic operations of the operation 220 or in the learning operation250. The assignment of weights to different groups of example datastrings help to ensure that the more important target data categorieshave the priority of being represented in the regular expressiongenerated by the genetic algorithm.

In some embodiments, the program generation module 122 operates togenerate a single regular expression for all the target data categoriesrepresented by the positive example data strings. The grouping of theexample data strings does not necessarily lead to the genetic operationsbeing operated on each group of example data strings separately.However, based on the final or intermediary results of the geneticoperations, the grouping of the example data strings may be adjusted andthe genetic operations may be adjusted accordingly, which are describedherein in detail.

In some embodiments, the initial grouping operation of the sub-act 412is omitted. The program generation module 122, by default, will firsttry to generate a single regular expression that is able to extract allthe target data categories represented by the example data strings. Theexample grouping unit 312 may group the example data strings based onthe inputs or feedback from the adjustment unit 352 later in theoperations of the genetic algorithm as described herein in detail.

In sub-act 414, the initial program generation unit 314 coordinates withthe random program generation unit 320 to generate an initial populationof candidate regular expressions, also referred to as “candidateprograms.” Specifically, in some embodiments, the initial programgeneration unit 314 generates candidate regular expressions based on thepositive example data strings. The random program generation unit 320generates candidate regular expressions randomly. In some embodiments, aratio between a number of candidate regular expressions generated basedon the positive example data strings and a number of randomly generatedcandidate regular expressions is maintained within a range between about1:7 to about 1:10. In some embodiments, a ratio between the number ofcandidate regular expressions generated based on the positive exampledata strings and the number of randomly generated candidate regularexpressions is 1:9. In some embodiments, the ratio is controlled by thecontroller 350. Experimental data shows that such a range of ratioshelps to ensure that a final regular expression exhibits an extractionbehavior that is consistent with those of the example data strings andfurther expends from those of the example data strings. In someembodiments, all the candidate regular expressions in the initialpopulation are randomly generated and the example data strings providedby the user 110 are used in the genetic operations 420, 430 as describedherein.

Regular expressions are typically described in a string describing thepattern it represents. A regular expression may include one or moreelements of a literal, e.g., \a\; a character range, e.g., \[a-z]\; anegated character range, e.g., \[{circumflex over ( )}a-z]\; aconcatenation, e.g., \a[bc]\; an option operator, e.g., \a?\; a staroperator, e.g., \a*\; a plus operator, e.g., \a+\; a non-greedyoperators, e.g., \a??\, \a *? \, \a+? \; an alternation operator, e.g.,\a|b \; or a capture group operator, e.g., \(ab) \.

For at least some of the positive example data strings, two or morecandidate regular expressions are generated based on each of them. As anillustrative example, an example data string of “175.8” can berepresented by regular expressions of:

r:=\d\d\d\.\d; or

r:=\d+\.\d.; or

r: =\[0-2]\d\d\.\d

Further, the candidate regular expressions generated for differentpositive example data strings may overlap. The overlapped regularexpressions may be filtered out from the initial population or may bekept in the initial population to increase the possibility that theextraction behaviors of such candidate regular expressions or the“genes” be properly represented in the initial population and besufficiently carried over in the evolution process.

In some embodiments, various strategies are considered in generating theinitial population of regular expressions based on the positive exampledata strings. For example, a strategy may favor simplified regularexpression over complicated regular expression. A strategy may tryreducing or increasing the function labels or types of function labelsused in a regular expression. Those strategies affect the final regularexpression generated through the genetic operations, which may beadjusted in the learning operation 250.

In some embodiments, the population size of the initial population ofcandidate regular expressions is larger than the total number of thepositive example data strings. For example, the population size is about1.5-2 times the number n of positive example data strings.

In some embodiments, in procedure 416 of the sub-act 414, the fitnessmeasurement unit 340 calculates a fitness score for each of thecandidate regular expressions in the initial population. The fitnessmeasurement unit 340 may use various suitable fitness algorithms tocalculate the fitness scores, which are all included in the scope of thespecification. In some embodiments, the fitness measurement unit 340uses the below algorithm to calculate the fitness scores:f(R)=Σ_(i=0) ^(n) d(s _(i) ,R(t _(i)))+α·l(R)+β·p ₊ −γ·p ⁻  (1),where, t_(i) denotes a positive example data string; n is a total numberof positive example data string; s_(i) denotes a snippet contained inthe positive example data string t_(i) that is a target data category;R(t_(i)) denotes a snippet extracted from the positive example datastring t_(i) by a regular expression; d(x₁,x₂) denotes an edit distancebetween data string/snippet x₁, x₂; l(R) denotes a length of a regularexpression; p₊ denotes a rate that a regular expression matches thepositive example data strings, e.g., R(t_(i))=s_(i); p⁻ denotes a ratethat a regular expression matches the negative example data string; andα, β, γ are constants, which can be adjusted by the controller 350. Insome embodiments, t_(i)=s_(i) because the positive example data stringsare exactly representative of target data categories. It should beappreciated that although a big portion of the initial population ofcandidate regular expressions are each generated directly from one ormore of the positive example data strings, they each may not be able toextract other positive example data strings. It should be appreciatedthat the data strings that are used for calculating the fitness scoremay be a different set or group of data strings from those used togenerate the initial candidate programs.

In some embodiments, n may be the number of all example data strings;t_(i) denotes an example data string; s_(i) denotes a snippet containedin the example data string t_(i) that is a target data category, whiles_(i)=0 for a negative example data string not containing a target datacategory; R(t_(i)) denotes a snippet extracted from the example datastring t_(i) by a regular expression; d(x₁,x₂) denotes an edit distancebetween data string/snippet x₁, x₂.

In some embodiments, β, γ are adjusted based on the error tolerance ofthe user 110. For example, if the user 110 balances toward falsepositive over false negative, β will be increased. If the user 110balances toward false negative over false positive, γ will be increased.

In some embodiments, weights assigned to a group of example data stringsmay be introduced to the fitness function.f(R)=Σ_(i=0) ^(n) w _(i) ·d(s _(i) ,R(t _(i)))+α·l(R)+β·Σ_(j=0) ^(m) w_(i) ·p _(+j)−γ·Σ_(j=0) ^(m) w _(i) ·p _(−j)  (2),where w_(i) is a weight of an example data string t_(i) that is equal tothe weight assigned to the group t_(i) belongs to; w_(j) is a weightassigned to a group j of example data strings; p_(+j) is a rate that aregular expression matches the positive example data string of group j;p_(−j) is a rate that a regular expression matches the negative exampledata string of group j; and m is a total number of groups of exampledata strings.

In sub-act 418, the parse tree unit 316 parses each candidate regularexpression in the initial G₀ generation. At least some of the candidateregular expressions are parsed into two or more components. In someembodiments, a parse tree is used to represent a regular expressionparsed into two or more components. In some embodiments, a symbolicfinite automata is used to represent a regular expression parsed intotwo or more components. Other approaches to represent a parsed regularexpression and/or the correspondence between the two or more componentsof a parsed regular expression are also possible and included in thescope of the specification. In the description herein, a parse tree isused as an example to illustrate the operation of the program generationmodule 122, which does not limit the scope of the specification.

In some embodiments, the parse tree is a constituency-based parse treethat includes terminal nodes and nonterminal nodes. Leave nodes(terminal nodes) of the parse tree are labeled with terminals,representing the components of the regular expression that have beenparsed. The leave nodes each does not have any children, and cannot beexpanded any further. When the leave nodes are concatenated together,the candidate regular expression is obtained. Each internal or non-leafnode (nonterminal node) of a parse tree is labeled with a nonterminallabel. A nonterminal label may include a placeholder label c and afunction label. The immediate children of the internal node must followa pattern of the function label's production rule in the grammar. Theplaceholder label c indicates a “place” of the associated child node. Afunction label indicates a functional relationship of a place c or afunctional relationship between/among multiple places c. For example(c1c2) indicates that child nodes associated to the two placeholders c1,c2 be concatenated; and {circumflex over ( )}c indicates that the childnode associated with the placeholder c be negated. In some embodiments,a string transformation approach is used form the parse tree. The stringtransformation of an internal node is achieved by replacing aplaceholder c with a string transformation result of the child nodeassociated with the placeholder c. Other approaches to form a parse treebased on a regular expression are also possible and included in thescope of the specification.

In example act 420, the synthesizing unit 330 synthesizes the candidateprograms in a parent Gp generation of candidate programs to produce achild Gp+1 generation of candidate programs. The synthesizing includescrossover and mutation operations on the parent generation of candidateprograms. A crossover operation interleaves two or more parent candidateprograms into two or more new candidate programs by recombiningcomponents or gene values of the two or more parent candidate programsto generate “child” candidate programs that each includes componentsfrom each parents. For example, in a case that parent candidate programsare represented as parse trees, the sub-trees or branches of the parentparse trees may be recombined to generate child parse trees. A mutationoperation alters one or more components or gene values of a parentcandidate program to generate a child candidate program. For example, ina case that a parent candidate program is represented as a parse tree, asub-tree or a branch of the parse tree may be replaced with a randomlygenerated sub-tree or branch to generate a child candidate program. Afunction of the mutation operation is to improve diversity of thepopulation of candidate programs.

In some embodiments, the fitness scores of candidate programs areconsidered in selecting the candidate programs for the mutation and thecrossover operations. For example, a candidate program's chance of beingselected to mate with another candidate program for the crossoveroperation may be in line with its fitness score. That is, a candidateprogram of a higher fitness score will have a higher chance to mate withanother candidate program in the crossover operation than a candidateprogram of a lower fitness score. As such, the “good gene,” e.g., thesuitable extraction behavior, can be carried over to the nextgeneration. In some embodiments, a candidate program of a lower fitnessscore will have a higher probability of being selected for the mutationoperation than being selected for the crossover operation. Thisincreases the chance that a “good gene” be introduced to the populationof candidate programs. In some embodiments, a candidate program of ahigher fitness score will have a higher probability of being selectedfor the mutation operation than a candidate program of a lower fitnessscore.

In some embodiments, the child Gp+1 generation of candidate program alsoincludes a small portion of randomly generated candidate programs by therandom program generation unit 320.

In some embodiments, the child Gp+1 generation of candidate programs hasa same number of candidate programs as the parent Gp generation ofcandidate programs. In a case where the synthesizing operationsinitially generate more candidate programs than the needed number, thegenerated candidate programs are filtered by their fitness scores. Thecandidate programs with lower fitness scores are filtered out until thechild Gp+1 generation of candidate programs has a same population sizeas the parent Gp generation.

In some embodiments, the child Gp+1 generation of candidate programsinclude a first subset of candidate programs that are generated throughthe crossover operation; a second subset of candidate programs that aregenerated through the mutation operation; and a third subset ofcandidate programs that are randomly generated. In some embodiments, aratio among the first subset, the second subset and the third subset ofcandidate programs in the number of candidate programs contained in eachsubset is maintained as substantially the same among all the generationsof candidate programs other than the initial generation G₀. For example,the ratio among the first subset, the second subset and the third subsetis in a range between 3:1:1 and 18:1:1. The size ratio may be controlledthrough the number of candidate programs generated through each of thecrossover operation, the mutation operation or random operation. Thesize ratio may also be controlled through selectively filtering out thecandidate programs with lower fitness scores in each subset.

In some embodiments, a ratio between the first subset of candidateprograms generated through the crossover operations and the secondsubset of candidate programs generated through the mutation operationsare determined based on the initial analysis of the example datastrings. For example, in a case that the example data strings are morehomogeneous, e.g., in the lengths of the example data strings or thedata categories represented by the example data strings, the size of thefirst subset will increase. In a case that the example data strings aremore heterogeneous, the size of the second subset will increase.

In example sub-act 422, the crossover unit 332 conducts the crossoveroperation. As illustrated in the example used herein, the candidateprograms are regular expressions and are each represented as parsetrees, which are suitable data structures for crossover operations. Thecrossover operation may be conducted in various ways of recombiningcomponents of the mated parent programs. For example, one or more of thesingle point crossover, two-point crossover (or k-point crossover), oruniform crossover may be used. Further, the function labels of the parsetree may be considered in the crossover operation. For example, one ormore of partially matched crossover, cycle crossover, order crossover,order-based crossover, position-based crossover, voting recombinationcrossover, alternating-position crossover, or sequential constructivecrossover may be used to properly handle the function labels in a parsetree.

In some embodiments, for a mated candidate program that is representedas a parse tree, a subtree/branch of the parse tree is randomly selectedfor recombination in a crossover operation. That is, when an internalnode is selected, the whole branch below the selected internal node,i.e., all the child nodes under the internal node, is used for therecombination in the crossover operation. In some other embodiments, anode of a parse tree is randomly selected, and only the selected node isused for recombination in a crossover operation. The child nodes, ifany, of the selected node will not be used for recombination.

In some embodiments, only the leaf nodes (or terminal nodes) of a parsetree are possibly selected for recombination in a crossover operation.The leaf nodes are randomly selected or are selected based on someconstraints. For example, the chance of a leaf node being selectedrelates to the distance between the leaf node and the root node of theparse tree. In some embodiments, a leaf node that is positioned fartherfrom the root node, e.g., more internal nodes therebetween, has a higherchance of being selected for recombination. In some other embodiments, aleaf node that is positioned farther from the root node has a lowerchance of being selected for recombination.

In some embodiments, a chance of an internal node being selected forrecombination is related to a height of the internal node, e.g., alongest distance between the internal node and a leaf node under theinternal node. For example, an internal node of a larger height may bemore likely to be selected for recombination. For another example, aninternal node of a larger height may be less likely to be selected forrecombination.

Other approaches of selecting nodes in a parse tree for recombinationare also possible and included in the scope of the specification. Insome embodiments, the approaches of selecting components of a candidateprogram for recombination can be configured and adjusted by thecontroller 350 as described herein.

In example sub-act 424, the mutation unit 334 conducts mutationoperations on candidate programs selected for the mutation operation. Asillustrated in the example used herein, the candidate programs areregular expressions and are each represented as parse trees, which aresuitable data structures for mutation operations. The mutation operationmay be conducted in various ways as controlled by the controller 350,which are all included in the scope of the specification. For example,one or more of the bit string mutation, flip bit mutation, boundarymutation, non-uniform mutation, uniform mutation, Gaussian mutation, orshrink mutation may be used.

In some embodiments, for a candidate program that is represented as aparse tree, a subtree/branch of the parse tree is randomly selected fora mutation operation. A randomly generated sub-tree or branch willreplace the selected subtree. That is, when an internal node isselected, the whole branch below the selected internal node, i.e., allthe child nodes under the internal node, is replaced by another subtreein the mutation operation. In some other embodiments, a node of a parsetree is randomly selected, and only the selected node is replaced byanother randomly generated node. For example, a function label of thenonterminal label may be replaced by a randomly generated functionlabel. The child nodes, if any, of the selected node will not be usedfor mutation.

In some embodiments, only the leaf nodes (or terminal nodes) of a parsetree are possibly selected for mutation. The leaf nodes are randomlyselected or are selected based on some constraints. For example, thechance of a leaf node being selected relates to the distance between theleaf node and the root node of the parse tree. In some embodiments, aleaf node that is positioned farther from the root node, e.g., moreinternal nodes therebetween, has a higher chance of being selected formutation. In some other embodiments, a leaf node that is positionedfarther from the root node has a lower odd of being selected formutation.

In some embodiments, a chance of an internal node being selected formutation relates to a height of the internal node, e.g., a longestdistance between the internal node and a leaf node under the internalnode. For example, an internal node of a larger height may be morelikely to be selected for mutation. For another example, an internalnode of a larger height may be less likely to be selected for mutation.

Other approaches of selecting nodes in a parse tree for mutationoperation are also possible and included in the scope of thespecification. In some embodiments, the approaches of selectingcomponents of a candidate program for the mutation can be configured andadjusted by the controller 350 as described herein.

In example sub-act 426, the random program generation unit 320 randomlygenerates candidate programs for the Gp+1 generation.

In example act 430, the fitness measurement unit 340 obtains fitnessscores for the candidate programs. In some embodiments, a same fitnessfunction, like the function (1) or the function (2), may be used toobtain fitness scores of the parent candidate programs of generation Gp,the child candidate programs in the generation Gp+1, and the initialcandidate programs in the G₀ generation. In some embodiments, differentfitness functions may be used. In some embodiments, a fitness functionincludes factors related to one or more factors of conciseness of acandidate regular expression (e.g., a length of the candidate regularexpression), a first matching rate of the candidate regular expressionon the positive example data strings, a second matching rate of thecandidate regular expression on the negative example data strings, or anedit distance between the candidate regular expression and an exampledata string.

In example sub-act 432, optionally, the fitness measurement unit 340filtered the new candidate programs based on the fitness scores thereof.For example, new candidate programs of a lower fitness score may beremoved from the population of the Gp+1 generation of the candidateprograms. In some embodiments, the filtering operations are conductedseparately for the first subset, the second subset and the third subsetof new candidate programs such that the size ratio among the firstsubset, the second subset and the third subset of new candidate programsis maintained.

The acts 420, 430 are together referred to as a round of geneticoperation or evolution. The genetic operation is conducted iteratively,with each round of the genetic operation or evolution generating a newgeneration of candidate programs. The controller 350 may set upthreshold conditions on completing or terminating the iterative geneticoperations. For example, a threshold condition includes that a totalnumber of the iteration reaches a threshold number or that a fitnessscore of a candidate program reaches a threshold fitness score. Athreshold condition may also include that a round of genetic operationgenerates no new benefits. A new benefit includes an improved fitnessscore, either individually or in average. A new benefit also includes anew candidate program that is different from any existing candidateprograms.

In example act 440, the controller 350 determines whether a thresholdcondition on completing the iterative genetic operations has been met.If none of the threshold conditions are met, the controller 350 controlsthe genetic operation to continue the iteration. If one or more of thethreshold conditions has been met, the controller 350 controls thegenetic operation to complete.

In example act 450, after the genetic operation is complete, the programgeneration module selects a candidate program that has the highestfitness score as the final pattern program. The final pattern program isoutput to the extraction module 124 to conduct the named entityrecognition tasks on the data streams 130.

FIG. 5 shows another example process 500. The process 500 includesadditional acts over the example process 400. The example acts 410, 420,430, 440 and 450 in the example process 500 are similar to those inprocess 400, the description of which are omitted with respect to theprocess 500 for simplicity purposes.

After a round of genetic operation, e.g., the acts 420 and 430, iscompleted, the controller 350 may route the operation to an act 510,where the adjustment unit 352 evaluates the candidate programs generatedin the round of genetic operation to determine whether the iterativegenetic operation should be adjusted. Specifically, in sub-act 512, theadjustment unit 352 obtains an average fitness score of all the childcandidate programs in the Gp+1 generation. The average fitness score iscompared to the average fitness score of parent Gp generation ofcandidate programs. If the average fitness score of the Gp+1 generationis smaller than the average fitness score of the Gp generation, theparameters of the genetic algorithm may be adjusted.

In sub-act 514, the adjustment unit 352 evaluates regrouping the exampledata strings. In some embodiments, the adjustment unit 352 analyzes eachpositive example data string regarding whether the positive example datastring matches a candidate program in the Gp+1 generation. A match rateis obtained for each positive example data string, which is calculatedas a number of matches between the positive example data string and thecandidate programs versus a total number of the candidate programs inthe Gp+1 generation. A threshold match rate may be set up, e.g., 50%match, by the controller 350. A positive example data string of a matchrate higher than the threshold match rate may be regrouped into a“conquered” group indicating that the extraction behaviors or “genes” ofthe candidate programs of the Gp+1 generation generally fit the specificpositive example data string. A positive example data string of a matchrate lower than the threshold match rate may be regrouped into an“unconquered” group indicating that the extraction behaviors or “genes”of the candidate programs of the Gp+1 generation generally do not matchwith the specific positive example data string. The conquered group ofpositive example data strings may be used in further genetic operations,e.g., in the calculation of fitness scores of candidate programs. Theunconquered group of positive example data strings may be used to obtainanother pattern program in a separate genetic operation.

In sub-act 516, the adjustment unit 352 evaluates regrouping thecandidate programs based on their extraction behaviors on differentgroups of positive example data strings. For example, the fitness scoresand/or the positive match rates of a candidate program are calculatedfor each group of example data strings. The candidate programs aregrouped based on their fitness scores or positive match rates for eachgroup of example data strings. For example, a candidate program may havea 70% match rate for a first group of positive example data strings anda 20% match rate for a second group of positive example data strings.The candidate program may be grouped as a candidate program suitable forextracting target data categories represented by the first group ofpositive example data strings. A group of candidate programs may be usedfor the genetic operations within the group. For example, a candidateprogram may only mate with another candidate program in the same groupfor the crossover operation.

The regrouping of the example data strings or the candidate programs maylead to that multiple genetic operations are conducted in parallel andmultiple final pattern programs are generated from the multiple geneticoperations. In some embodiments, the multiple final pattern programs maybe linked through “OR” functions in an extraction task.

In example act 520, the controller 350 determines whether an adjustmentof the genetic operation should be made based on the evaluation resultsof the act 510. If it is determined that one or more adjustments shouldbe made, the controller 350 effects the adjustment on the act 410 or theact 420. For example, the regrouping of the positive example datastrings may be used to adjust the genetic operations starting at the act420. For example, multiple genetic operations start to run in parallel.The conquered group and unconquered group of the positive example datastrings may also be used to reshape the initial population of candidateprograms at the act 410. Other approaches of adjusting the operation ofthe program generation module 122 are also possible and included in thescope of the specification.

The learning unit 360 is configured to function together with thecontroller 350 in further training the genetic operations. For example,the training data, e.g., the correct extraction results and theincorrect extraction results, may be used as training data strings. Ageneration of the candidate programs may be selected as an initialtraining population of candidate programs to start the trainingoperation. In some embodiments, the last generation of the candidateprograms is used as an initial training population. In some embodiments,the processes 400, 500 of FIG. 4 or 5 may be similarly run using thetraining data strings and on the initial training population ofcandidate programs. The training operation generates a new final patternprogram(s) that overcomes the shortcomings of the previous patternprograms in extracting the incorrect data strings.

The system, apparatus, module, or unit illustrated in the previousembodiments can be implemented by using a computer chip or an entity, orcan be implemented by using a product having a certain function. Atypical embodiment device is a computer, and the computer can be apersonal computer, a laptop computer, a cellular phone, a camera phone,a smartphone, a personal digital assistant, a media player, a navigationdevice, an email receiving and sending device, a game console, a tabletcomputer, a wearable device, or any combination of these devices.

For an embodiment process of functions and roles of each module in theapparatus, references can be made to an embodiment process ofcorresponding steps in the previous method. Details are omitted here forsimplicity.

Because an apparatus embodiment basically corresponds to a methodembodiment, for related parts, references can be made to relateddescriptions in the method embodiment. The previously describedapparatus embodiment is merely an example. The modules described asseparate parts may or may not be physically separate, and partsdisplayed as modules may or may not be physical modules, may be locatedin one position, or may be distributed on a number of network modules.Some or all of the modules can be selected based on actual demands toachieve the objectives of the solutions of the specification. A personof ordinary skill in the art can understand and implement theembodiments of the present application without creative efforts.

The techniques described in this specification produce one or moretechnical effects. The genetic algorithm works on data strings that eachexactly represents the target data categories of named entityrecognition. These technical features bring about valuable technicaladvantages. First, a pattern program generated from the geneticalgorithm will have a tailored extraction behavior because the good“genes” contained in the example data strings are efficiently caught andcarried over through the genetic operations of the genetic algorithm. Assuch, the generated pattern program will correctly detect and extractdata strings of the target data categories. Further, using such exampledata strings also reduces human inputs and errors in the process becausethere is no need to manually identify named entities from anunrepresentative data string. Also, the initial population of patternprograms is generated largely, e.g., 90%, from the example data strings,which substantially reduces the amount of iterative genetic operationsrequired to achieve a satisfactory pattern program. This saving incomputing resources is critical in managing large scale data streams inthe era of big data and cloud-based data services.

Moreover, the fitness function considers whether a pattern programmatches a negative example data string, which is not a target of a namedentity recognition task. Resultantly, a pattern program selected basedon the fitness function will avoid data categories that are representedby the negative example data strings. Therefore, the false positiveerrors will be substantially reduced, which makes the outcomes of thenamed entity recognition tasks more reliable and meaningful. As such,the techniques of the specification are efficient and suitable forconducting named entity recognition tasks on large scale data streams.

The techniques operate on various example data strings withoutdifferentiating among the various example data strings, which generate apattern program that functions to extract target data categoriesrepresented by all the example data strings. As such, the operation ofthe techniques can be done fully autonomously without humanintervention. If the initial efforts of generating a single patternprogram fails, the example data strings can be regrouped and the geneticoperation parameters can be adjusted based on the evaluation of theprevious operation results without human intervention. As such, thetechniques generate a computer program, e.g., a regular expression,fully autonomously based on example data strings representative of thedata categories to be matched by the regular expression.

Described embodiments of the subject matter can include one or morefeatures, alone or in combination. For example, in a first embodiment, acomputer-implemented method obtains a first population of candidateprograms; generates a second population of candidate programs byconducting an iterative genetic operation on the first population ofcandidate programs; and extracts a second plurality of data strings froma data stream using a first candidate program of the second populationof candidate programs. The iterative genetic operation includescalculating a fitness score for each candidate program of the secondpopulation of candidate programs using a fitness function and a firstplurality of data strings. The fitness function evaluates a match rateof a candidate program with the first plurality of data strings;

The foregoing and other described embodiments can each, optionally,include one or more of the following features.

A first feature, combinable with any of the previous or followingfeatures, specifies that the method further includes obtaining a thirdplurality of data strings. The third plurality of data string is asubset of the second plurality of data strings. The method furtherincludes generating a second candidate program by conducting theiterative genetic operation on the second population of candidateprograms using the third plurality of data strings.

A second feature, combinable with any of the previous or followingfeatures, specifies that the first plurality of data strings include aplurality of positive example data strings each representative of atarget data category of a named entity recognition task.

A third feature, combinable with any of the previous or followingfeatures, specifies that the first plurality of data strings include aplurality of negative example data strings each representative of a datacategory that negates the target data category.

A fourth feature, combinable with any of the previous or followingfeatures, specifies that the fitness function evaluates a first matchrate of the candidate program with respect to the plurality of positiveexample data strings and a second match rate of the candidate programwith respect to the plurality of negative example data strings.

A fifth feature, combinable with any of the previous or followingfeatures, specifies that the method further includes grouping the firstplurality of data strings into a first group of data strings and atleast one second group of data strings, and separately conducting theiterative genetic operation on the first set of candidate programs andusing each of the first group of data strings or the at least one secondgroup of data strings.

A sixth feature, combinable with any of the previous or followingfeatures, specifies that the fitness function further evaluates aconciseness of the candidate program and an edit distance between thecandidate program and a data string of the first plurality of datastrings.

A seventh feature, combinable with any of the previous or followingfeatures, specifies that the iterative genetic operation includes acrossover operation and a mutation operation.

A eighth feature, combinable with any of the previous or followingfeatures, specifies that a candidate program of each of the firstpopulation of candidate programs or the second population of candidateprograms is a regular expression.

A ninth feature, combinable with any of the previous or followingfeatures, specifies that each of the first plurality of data strings isassigned with a weight; and the fitness function evaluates the weight ofeach of the first plurality of data strings.

A tenth feature, combinable with any of the previous or followingfeatures, specifies that the first candidate program has a highestfitness score among the second population of candidate programs; theconducting the iterative genetic operation on the second population ofcandidate programs using the third plurality of data strings generates athird population of candidate programs; and the second candidate programhas a highest fitness score among the third population of candidateprograms.

An eleventh feature, combinable with any of the previous or followingfeatures, specifies that the second candidate program has a fitnessscore that is higher than a fitness score of the first candidate programcalculated using at least one of the first plurality of data strings andthe third plurality of data strings.

A twelfth feature, combinable with any of the previous or followingfeatures, specifies that the obtaining the first population of candidateprograms includes obtaining at least a portion of the first populationof candidate programs based on the first plurality of data strings.

In a second embodiment, a system comprises one or more processors, andone or more computer-readable memories coupled to the one or moreprocessors and having instructions stored thereon that are executable bythe one or more processors to perform acts. The acts include receiving,from a user, a first plurality of data strings; obtaining a firstpopulation of candidate programs at least partially based on the firstplurality of data strings; generating a second population of candidateprograms by conducting an iterative genetic operation on the firstpopulation of candidate programs, the iterative genetic operationincluding calculating a fitness score for each candidate program of thesecond population of candidate programs using a fitness function and thefirst plurality of data strings; extracting a second plurality of datastrings from a data stream using a first candidate program selected fromthe second population of candidate programs; providing the secondplurality of data strings to the user; receiving a third plurality ofdata strings from the user, the third plurality of data strings being asubset of the second plurality of data strings; and obtaining a secondcandidate program at least partially based on the third plurality ofdata strings and the second population of candidate programs.

In a third embodiment, an apparatus comprises a plurality of modules andunits. The plurality of modules and units include an initial programgeneration unit that operates to obtain a first population of candidateprograms; a synthesizing unit that operates to generate a secondpopulation of candidate programs by conducting an iterative geneticoperation on the first population of candidate programs; a fitnessmeasurement unit that operates to calculate a fitness score for eachcandidate program of the second population of candidate programs using afitness function and a first plurality of data strings, the fitnessfunction evaluating a match rate of a candidate program with the firstplurality of data strings; and an extraction module that operates toextract a second plurality of data strings from a data stream using afirst candidate program selected from the first population of candidateprograms and the second population of candidate programs.

In a fourth embodiment, a non-transitory computer-readable storagemedium stores executable instructions that cause a processor to executeacts comprising: obtaining a first population of candidate programs;generating a second population of candidate programs by conducting aniterative genetic operation on the first population of candidateprograms, the iterative genetic operation including calculating afitness score for each candidate program of the second population ofcandidate programs using a fitness function and a first plurality ofdata strings; based on fitness scores of candidate programs of thesecond population of candidate programs, dividing the first plurality ofdata strings into a first subset of data strings and at least one secondsubset of data strings; generating a third population of candidateprograms by conducting the iterative genetic operation on the secondpopulation of candidate programs using the first subset of data strings;and extracting a second plurality of data strings from a data streamusing a first candidate program selected from the third population ofcandidate programs.

Embodiments of the subject matter and the actions and operationsdescribed in this specification can be implemented in digital electroniccircuitry, in tangibly-embodied computer software or firmware, incomputer hardware, including the structures disclosed in thisspecification and their structural equivalents, or in combinations ofone or more of them. Embodiments of the subject matter described in thisspecification can be implemented as one or more computer programs, e.g.,one or more modules of computer program instructions, encoded on acomputer program carrier, for execution by, or to control operation of,data processing apparatus. For example, a computer program carrier caninclude one or more computer-readable storage media that haveinstructions encoded or stored thereon. The carrier may be a tangiblenon-transitory computer-readable medium, such as a magnetic, magnetooptical or optical disk, a solid state drive, a random access memory(RAM), a read-only memory (ROM), or other types of media. Alternatively,or in addition, the carrier may be an artificially generated propagatedsignal, e.g., a machine-generated electrical, optical, orelectromagnetic signal that is generated to encode information fortransmission to suitable receiver apparatus for execution by a dataprocessing apparatus. The computer storage medium can be or be part of amachine-readable storage device, a machine-readable storage substrate, arandom or serial access memory device, or a combination of one or moreof them. A computer storage medium is not a propagated signal.

A computer program, which may also be referred to or described as aprogram, software, a software application, an app, a module, a softwaremodule, an engine, a script, or code, can be written in any form ofprogramming language, including compiled or interpreted languages, ordeclarative or procedural languages; and it can be deployed in any form,including as a stand-alone program or as a module, component, engine,subroutine, or other unit suitable for executing in a computingenvironment, which environment may include one or more computersinterconnected by a data communication network in one or more locations.

A computer program may, but need not, correspond to a file in a filesystem. A computer program can be stored in a portion of a file thatholds other programs or data, e.g., one or more scripts stored in amarkup language document, in a single file dedicated to the program inquestion, or in multiple coordinated files, e.g., files that store oneor more modules, sub programs, or portions of code.

Processors for execution of a computer program include, by way ofexample, both general- and special-purpose microprocessors, and any oneor more processors of any kind of digital computer. Generally, aprocessor will receive the instructions of the computer program forexecution as well as data from a non-transitory computer-readable mediumcoupled to the processor.

The term “data processing apparatus” encompasses all kinds ofapparatuses, devices, and machines for processing data, including by wayof example a programmable processor, a computer, or multiple processorsor computers. Data processing apparatus can include special-purposelogic circuitry, e.g., an FPGA (field programmable gate array), an ASIC(application specific integrated circuit), or a GPU (graphics processingunit). The apparatus can also include, in addition to hardware, codethat creates an execution environment for computer programs, e.g., codethat constitutes processor firmware, a protocol stack, a databasemanagement system, an operating system, or a combination of one or moreof them.

The processes and logic flows described in this specification can beperformed by one or more computers or processors executing one or morecomputer programs to perform operations by operating on input data andgenerating output. The processes and logic flows can also be performedby special-purpose logic circuitry, e.g., an FPGA, an ASIC, or a GPU, orby a combination of special-purpose logic circuitry and one or moreprogrammed computers.

Computers suitable for the execution of a computer program can be basedon general or special-purpose microprocessors or both or any other kindof central processing unit. Generally, a central processing unit willreceive instructions and data from a read only memory or a random accessmemory or both. Elements of a computer can include a central processingunit for executing instructions and one or more memory devices forstoring instructions and data. The central processing unit and thememory can be supplemented by, or incorporated in, special-purpose logiccircuitry.

Generally, a computer will also include, or be operatively coupled toreceive data from or transfer data to one or more storage devices. Thestorage devices can be, for example, magnetic, magneto optical, oroptical disks, solid state drives, or any other type of non-transitory,computer-readable media. However, a computer need not have such devices.Thus, a computer may be coupled to one or more storage devices, such as,one or more memories, that are local and/or remote. For example, acomputer can include one or more local memories that are integralcomponents of the computer, or the computer can be coupled to one ormore remote memories that are in a cloud network. Moreover, a computercan be embedded in another device, e.g., a mobile telephone, a personaldigital assistant (PDA), a mobile audio or video player, a game console,a Global Positioning System (GPS) receiver, or a portable storagedevice, e.g., a universal serial bus (USB) flash drive, to name just afew.

Components can be “coupled to” each other by being commutatively, suchas electrically or optically, connected to one another, either directlyor via one or more intermediate components. Components can also be“coupled to” each other if one of the components is integrated into theother. For example, a storage component that is integrated into aprocessor (e.g., an L2 cache component) is “coupled to” the processor.

To provide for interaction with a user, embodiments of the subjectmatter described in this specification can be implemented on, orconfigured to communicate with, a computer having a display device,e.g., a LCD (liquid crystal display) monitor, for displaying informationto the user, and an input device by which the user can provide input tothe computer, e.g., a keyboard and a pointing device, e.g., a mouse, atrackball or touchpad. Other kinds of devices can be used to provide forinteraction with a user as well, for example, feedback provided to theuser can be any form of sensory feedback, e.g., visual feedback,auditory feedback, or tactile feedback; and input from the user can bereceived in any form, including acoustic, speech, or tactile input. Inaddition, a computer can interact with a user by sending documents toand receiving documents from a device that is used by the user, forexample, by sending web pages to a web browser on a user's device inresponse to requests received from the web browser, or by interactingwith an app running on a user device, e.g., a smartphone or electronictablet. Also, a computer can interact with a user by sending textmessages or other forms of message to a personal device, e.g., asmartphone that is running a messaging application, and receivingresponsive messages from the user in return.

This specification uses the term “configured to” in connection withsystems, apparatus, and computer program components. For a system of oneor more computers to be configured to perform particular operations oractions means that the system has installed on it software, firmware,hardware, or a combination of them that in operation cause the system toperform operations or actions. For one or more computer programs to beconfigured to perform particular operations or actions means that theone or more programs include instructions that, when executed by dataprocessing apparatus, cause the apparatus to perform operations oractions. For special-purpose logic circuitry to be configured to performparticular operations or actions means that the circuitry has electroniclogic that performs operations or actions.

While this specification contains many specific embodiment details,these should not be construed as limitations on the scope of what isbeing claimed, which is defined by the claims themselves, but rather asdescriptions of features that may be specific to particular embodiments.Certain features that are described in this specification in the contextof separate embodiments can also be realized in combination in a singleembodiment. Conversely, various features that are described in thecontext of a single embodiment can also be realized in multipleembodiments separately or in any suitable subcombination. Moreover,although features may be described above as acting in certaincombinations and even initially be claimed as such, one or more featuresfrom a claimed combination can in some cases be excised from thecombination, and the claim may be directed to a subcombination orvariation of a subcombination.

Similarly, while operations are depicted in the drawings and recited inthe claims in a particular order, this should not be understood asrequiring that such operations be performed in the particular ordershown or in sequential order, or that all illustrated operations beperformed, to achieve desirable results. In certain circumstances,multitasking and parallel processing may be advantageous. Moreover, theseparation of various system modules and components in the embodimentsdescribed above should not be understood as requiring such separation inall embodiments, and it should be understood that the described programcomponents and systems can generally be integrated together in a singlesoftware product or packaged into multiple software products.

Particular embodiments of the subject matter have been described. Otherembodiments are within the scope of the following claims. For example,the actions recited in the claims can be performed in a different orderand still achieve desirable results. As one example, the processesdepicted in the accompanying figures do not necessarily require theparticular order shown, or sequential order, to achieve desirableresults. In some cases, multitasking and parallel processing may beadvantageous.

We claim:
 1. A computer-implemented method, the method comprising:obtaining a first population of candidate programs; generating a secondpopulation of candidate programs by conducting an iterative geneticoperation on the first population of candidate programs, the iterativegenetic operation including calculating a fitness score for eachcandidate program of the second population of candidate programs using afitness function and a first plurality of data strings, the fitnessfunction evaluating a match rate of a candidate program with the firstplurality of data strings; and extracting a second plurality of datastrings from a data stream using a first candidate program of the secondpopulation of candidate programs, wherein the first plurality of datastrings include a plurality of positive example data strings eachrepresentative of a target data category of a named entity recognitiontask.
 2. The method of claim 1, further comprising: obtaining a thirdplurality of data strings, the third plurality of data strings being asubset of the second plurality of data strings; and generating a secondcandidate program by conducting the iterative genetic operation on thesecond population of candidate programs using the third plurality ofdata strings.
 3. The method of claim 1, wherein the first plurality ofdata strings include a plurality of negative example data strings eachrepresentative of a data category that negates the target data category.4. The method of claim 3, wherein the fitness function evaluates a firstmatch rate of the candidate program with respect to the plurality ofpositive example data strings and a second match rate of the candidateprogram with respect to the plurality of negative example data strings.5. The method of claim 1, further comprising: grouping the firstplurality of data strings into a first group of data strings and atleast one second group of data strings; and separately conducting theiterative genetic operation on the first population of candidateprograms and using each of the first group of data strings or the atleast one second group of data strings.
 6. The method of claim 1,wherein the fitness function further evaluates a conciseness of thecandidate program and an edit distance between the candidate program anda data string of the first plurality of data strings.
 7. The method ofclaim 1, wherein the iterative genetic operation includes a crossoveroperation and a mutation operation.
 8. The method of claim 1, wherein acandidate program of each of the first population of candidate programsor the second population of candidate programs is a regular expression.9. The method of claim 1, wherein: each of the first plurality of datastrings is assigned with a weight; and the fitness function evaluatesthe weight of each of the first plurality of data strings.
 10. Themethod of claim 2, wherein: the first candidate program has a highestfitness score among the second population of candidate programs; theconducting the iterative genetic operation on the second population ofcandidate programs using the third plurality of data strings generates athird population of candidate programs; and the second candidate programhas a highest fitness score among the third population of candidateprograms.
 11. The method of claim 10, wherein the second candidateprogram has a fitness score that is higher than a fitness score of thefirst candidate program calculated using at least one of the firstplurality of data strings and the third plurality of data strings. 12.The method of claim 1, wherein the obtaining the first population ofcandidate programs includes obtaining at least a portion of the firstpopulation of candidate programs based on the first plurality of datastrings.
 13. A system, comprising: one or more processors; and one ormore computer-readable memories coupled to the one or more processorsand having instructions stored thereon that are executable by the one ormore processors to perform acts including: obtaining a first populationof candidate programs; generating a second population of candidateprograms by conducting an iterative genetic operation on the firstpopulation of candidate programs, the iterative genetic operationincluding calculating a fitness score for each candidate program of thesecond population of candidate programs using a fitness function and afirst plurality of data strings, the fitness function evaluating a matchrate of a candidate program with the first plurality of data strings;and extracting a second plurality of data strings from a data streamusing a first candidate program of the second population of candidateprograms, wherein the first plurality of data strings include aplurality of positive example data strings each representative of atarget data category of a named entity recognition task.
 14. The systemof claim 13, wherein the acts include: obtaining a third plurality ofdata strings, the third plurality of data strings being a subset of thesecond plurality of data strings; and generating a second candidateprogram by conducting the iterative genetic operation on the secondpopulation of candidate programs using the third plurality of datastrings.
 15. The system of claim 13, wherein the first plurality of datastrings include a plurality of negative example data strings eachrepresentative of a data category that negates the target data category.16. The system of claim 15, wherein the fitness function evaluates afirst match rate of the candidate program with respect to the pluralityof positive example data strings and a second match rate of thecandidate program with respect to the plurality of negative example datastrings.
 17. The system of claim 13, wherein the acts further comprise:grouping the first plurality of data strings into a first group of datastrings and at least one second group of data strings; and separatelyconducting the iterative genetic operation on the first population ofcandidate programs and using each of the first group of data strings orthe at least one second group of data strings.
 18. An apparatus, theapparatus comprising: an initial group generation unit that obtains afirst population of candidate programs; a synthesizing unit thatgenerates a second population of candidate programs by conducting aniterative genetic operation on the first population of candidateprograms, the iterative genetic operation including calculating afitness score for each candidate program of the second population ofcandidate programs using a fitness function and a first plurality ofdata strings, the fitness function evaluating a match rate of acandidate program with the first plurality of data strings; and anextraction module that extracts a second plurality of data strings froma data stream using a first candidate program of the second populationof candidate programs, wherein the first plurality of data stringsinclude a plurality of positive example data strings each representativeof a target data category of a named entity recognition task.
 19. Theapparatus of claim 18, wherein the fitness function evaluates aconciseness of the candidate program and an edit distance between thecandidate program and a data string of the first plurality of datastrings.
 20. The apparatus of claim 18, wherein the first plurality ofdata strings include a plurality of negative example data strings eachrepresentative of a data category that negates the target data category;and wherein the fitness function evaluates a first match rate of thecandidate program with respect to the plurality of positive example datastrings and a second match rate of the candidate program with respect tothe plurality of negative example data strings.